Video delivery terminal, non-transitory computer-readable medium, and video delivery method

ABSTRACT

A video delivery terminal includes: an image capturer configured to perform image capture to generate an image of a subject; and a developing processor configured to crop an area of a portion of the image to generate a cropped image, perform first adjustment regarding the cropped image based on characteristics of the cropped image, and perform second adjustment regarding settings of the image based on characteristics of the image having undergone the first adjustment.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. § 119 toJapanese Patent Application No. 2015-148077, filed Jul. 27, 2015. Thecontents of which are incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to a video delivery terminal, anon-transitory computer-readable medium, and a video delivery method.

2. Description of the Related Art

Video delivery systems, such as video conferencing systems, in whicheach of remote sites displays an image(s) of the other site(s) bytransferring images between the remote sites, are known. Some type ofsuch video delivery systems displays a cropped image, which is an imagecropped from a wide-angle image taken with a wide-angle camera of avideo delivery terminal. An example of a video delivery system of thistype is disclosed in Japanese Unexamined Patent Application PublicationNo. 2006-222816 (Patent Document 1).

However, such a video delivery system has a problem that, becausesettings for exposure and the like of a cropped image cropped from awide-angle image are set with reference to settings for exposure and thelike of the other areas of the wide-angle image, the settings can beinappropriate.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, there is provided avideo delivery terminal including: an image capturer configured toperform image capture to generate an image of a subject; and adeveloping processor configured to crop an area of a portion of theimage to generate a cropped image, perform first adjustment regardingthe cropped image based on characteristics of the cropped image, andperform second adjustment regarding settings of the image based oncharacteristics of the image having undergone the first adjustment.

According to another aspect of the present invention, there is provideda non-transitory computer-readable medium including computer readableprogram codes, performed by a processor, the program codes when executedcausing the processor to execute: performing image capture to generatean image of a subject; cropping an area of a portion of the image togenerate a cropped image; performing first adjustment regarding thecropped image based on characteristics of the cropped image; andperforming second adjustment regarding settings of the image based oncharacteristics of the image having undergone the first adjustment.

According to still another aspect of the present invention, there isprovided a video delivery method including: performing image capture togenerate an image of a subject; cropping an area of a portion of theimage to generate a cropped image; performing first adjustment regardingthe cropped image based on characteristics of the cropped image; andperforming second adjustment regarding settings of the image based oncharacteristics of the image having undergone the first adjustment.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic overall diagram of a video delivery systemaccording to an embodiment;

FIG. 2 is a block diagram describing a control system of a terminal;

FIG. 3 is a block diagram describing a camera and image data processingin the terminal;

FIG. 4 is a flowchart of a process for calculating an evaluative meteredvalue of a cropped image of a speaker;

FIG. 5 is a diagram describing a cropped image of a speaker;

FIG. 6 is a diagram describing a weighting table for assigning weightsto Y signals of the cropped image;

FIG. 7 is a flowchart of an exposure control process performed on thecropped image of the speaker based on an evaluative metered value of thecropped image;

FIG. 8 is a flowchart of a process for calculating an evaluative meteredvalue of a wide-angle image;

FIG. 9 is a diagram describing the wide-angle image;

FIG. 10 is a flowchart of an exposure control process performed on thewide-angle image based on an evaluative metered value of the wide-angleimage;

FIG. 11 is a histogram describing an example of exposure control for thewide-angle image;

FIG. 12 is a histogram stretched to a region where Y signals (luminancevalues) are low in the exposure control for the wide-angle image; and

FIG. 13 is a diagram describing an example of a method for stretchingthe histogram.

The accompanying drawings are intended to depict exemplary embodimentsof the present invention and should not be interpreted to limit thescope thereof. Identical or similar reference numerals designateidentical or similar components throughout the various drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentinvention.

As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise.

In describing preferred embodiments illustrated in the drawings,specific terminology may be employed for the sake of clarity. However,the disclosure of this patent specification is not intended to belimited to the specific terminology so selected, and it is to beunderstood that each specific element includes all technical equivalentsthat have the same function, operate in a similar manner, and achieve asimilar result.

An embodiment of the present invention will be described in detail belowwith reference to the drawings.

The present invention has an object to provide a video deliveryterminal, a non-transitory computer-readable medium, and a videodelivery method capable of displaying a cropped image with appropriatesettings.

The following exemplary embodiments and modifications may include likeelements. In the following, like reference numerals denote likeelements, and repeated description may be partially omitted. A portionincluded in one of the embodiments and modifications can be replacedwith a corresponding portion in another one of the embodiments andmodifications. Configurations, positions, and the like of portionsincluded in the embodiments and modifications are similar to those ofthe other embodiments and modifications unless otherwise specificallystated.

Embodiment

FIG. 1 is a schematic overall diagram of a video delivery system 10according to an embodiment. The video delivery system 10 according tothe embodiment is applied to a video conferencing system that allowsusers at a plurality of remote sites to have a conference while viewingimages of the other user(s). As illustrated in FIG. 1, the videodelivery system 10 includes a server 12 and a plurality of terminals 14,each of which is an example of “video delivery terminal”. In thefollowing, if it is unnecessary to individually differentiate theterminals 14 a, 14 b, 14 c, and 14 d, the terminals are denoted by areference numeral “14”. Each of the video delivery terminals may beeither a dedicated video delivery terminal or an information processingterminal, such as a personal computer, a smartphone, and a tabletcomputer.

The server 12 is coupled to each of the terminals 14 via a network, suchas the Internet. The server 12 monitors whether or not each of theterminals 14 is in a state coupled to the server 12; furthermore, theserver 12 performs control for calling up the terminals 14 at start of aconference and a variety of control during the conference.

The terminal 14 transmits and receives data to and from the otherterminals 14 via the server 12. For example, the terminal 14 transmitsimage data and voice data captured by the terminal 14 to the server 12.The terminal 14 also receives, from the server 12, image data and voicedata transmitted by another one of the terminals 14. The terminal 14displays an image from the image data acquired from the other one of theterminals 14 and outputs voice from the voice data.

For example, in a situation where the terminals 14 a, 14 b, and 14 cparticipate in a conference but the terminal 14 d does not participatein the conference, data transmitted from the terminal 14 a is deliveredto the terminals 14 b and 14 c via the server 12 but not to the terminal14 d. Data transmitted from the terminal 14 b, 14 c is delivered to theterminal 14 a via the server 12 but not to the terminal 14 d. The videodelivery system 10 thus implements video conferencing with the pluralityof terminals 14.

FIG. 2 is a block diagram describing a control system of the terminal14. As illustrated in FIG. 2, the terminal 14 includes a camera 20,which is an example of “image capturer”, a display section 22, amicrophone 24, a loudspeaker 26, a CPU (Central Processing Unit) 28, astorage device 30, a memory 32, a LAN (Local Area Network) interfacesection 34, and an operating section 36.

The camera 20 generates image data representing a captured image of asubject, such as a participant of a conference. The camera 20 isconfigured to be capable of capturing an image of a wide-angle (e.g.,360 degrees) area. For example, when configured to capture an image ofan ultra-wide angle of 360 degrees, the camera 20 includes a pluralityof image sensors and lenses, which respectively correspond to the imagesensors. The camera 20 outputs the generated image data representing thewide-angle image to the CPU 28.

The display section 22 is, for example, a display device, such as aliquid crystal display or an organic electroluminescent (EL) display.The display section 22 displays an image from the image data acquiredfrom the CPU 28. For example, the display section 22 displays an imageof a conference participant near another one of the terminals 14 fromimage data acquired by the CPU 28 from the other one of the terminals14.

The microphone 24 is a microphone array, for example. The microphone 24converts captured voice into electrical voice data and outputs the voicedata to the CPU 28. The microphone 24 captures voice of a conferenceparticipant, for example.

The loudspeaker 26 outputs voice from voice data acquired from the CPU28. The loudspeaker 26 outputs voice of a conference participantcaptured by the microphone 24 of another one of the terminals 14, forexample.

The CPU 28 performs centralized control of the terminal 14. The CPU 28performs, for example, video-conference-related control includingtransmission and reception of image data and voice data in a videoconference.

The storage device 30 is a storage device, such as an HDD (Hard DiskDrive) and a non-volatile flash memory. The storage device 30 isconfigured such that the CPU 28 can read and write program instructionand data from and to the storage device 30. Specifically, the storageunit 30 stores program instructions to be executed by the CPU 28 anddata necessary for executing the program instructions. The storage unit30 stores, for example, program instructions for video conferencing andimage data necessary for executing the program instructions for thevideo conferencing.

The memory 32 is a RAM (Random Access Memory), for example. The memory32 is configured such that the CPU 28 can read and write programinstruction and data from and to the memory 32. The memory 32temporarily stores program instructions loaded therein for execution bythe CPU 28 and data, such as operations data.

The LAN interface section 34 is, for example, a wired LAN device that iscoupled to Ethernet (registered trademark) compliant with 10Base-T,100Base-TX, and 1000Base-T standards, or a wireless LAN device compliantwith 802.11a/b/g/n standards. The LAN interface section 34 is coupled tothe other terminals 14 via a network, the server 12, and the like in amanner that allows data transmission and reception. Hence, the LANinterface section 34 transmits and receives image data, voice data, andthe like necessary for video conferencing to and from the otherterminals 14 and outputs them to the CPU 28.

The operating section 36 is an input device, such as a keyboard, amouse, and a button, for example. The operating section 36 acquires aninstruction and the like entered by a user, e.g., a conferenceparticipant, for device control and outputs the instruction to the CPU28.

In the terminal 14 during a video conference, when the CPU 28 acquiresvoice data representing voice of a conference participant from themicrophone 24, the CPU 28 detects a direction, in which a speaker, i.e.,the speaking participant, is positioned, from the voice data. The CPU 28identifies the speaker of the voice data from a plurality ofparticipants contained in a wide-angle image represented by image dataacquired from the camera 20. The CPU 28 generates image datarepresenting a cropped close-up image of the identified speaker. The CPU28 corrects the wide-angle image containing the plurality ofparticipants and the cropped image of the speaker as necessary.

The CPU 28 transmits the image data representing the wide-angle imagecontaining the plurality of participants, the image data representingthe cropped close-up image of the speaker, voice data of theparticipants including the speaker, and the like to the server 12 viathe LAN interface section 34 and the network.

The CPU 28 receives, from the server 12, image data and voice datatransmitted from another one of the terminals 14 via the LAN interfacesection 34 and the network. The CPU 28 outputs the received image datato the display section 22 and causes the display section 22 to displayparticipants and a speaker. Furthermore, the CPU 28 outputs the voicedata to the loudspeaker 26 and causes the loudspeaker 26 to reproducevoice of the speaker. The terminals 14 perform video conferencing inthis way.

FIG. 3 is a block diagram describing the camera 20 and image dataprocessing in the terminal 14. As illustrated in FIG. 3, the camera 20of the terminal 14 includes a lens 40, an aperture 42, and an imagesensor 44. The terminal 14 further includes a developing processor 46.For example, the CPU 28 having loaded program instructions thereon mayfunction as the developing processor 46.

The image sensor 44 is, for example, a CMOS (Complementary Metal OxideSemiconductor) image sensor or a CCD (Charge-Coupled Device) imagesensor. The image sensor 44 receives light condensed by the lens 40 andpassed through the aperture 42, by which the amount of light isadjusted. The image sensor 44 performs photoelectric conversion toconvert the received light into an electrical signal. The image sensor44 generates a raw signal by reading out and amplifying the electricalsignal and performing A/D conversion on the electrical signal. The imagesensor 44 outputs image data of the raw signal to the developingprocessor 46. For example, the image sensor 44 outputs image datarepresenting a wide-angle image containing plurality of conferenceparticipants to the developing processor 46.

The developing processor 46 generates a cropped image by cropping a partof an image, such as a wide-angle image. The developing processor 46performs first adjustment regarding the cropped image based oncharacteristics of the cropped image. For example, the developingprocessor 46 generates a cropped image by cropping an area of andsurrounding a speaker (hereinafter, “speaker's area”) contained in awide-angle image. An example of the characteristics of the cropped imageis an evaluative metered value of the cropped image, which will bedescribed later. Examples of the first adjustment include adjustment, orautomatic adjustment, of exposure time of the camera 20, adjustment, orautomatic adjustment, of the aperture 42 of the camera 20, gainadjustment or automatic gain adjustment, contrast adjustment orautomatic contrast adjustment, adjustment, or automatic adjustment, ofgamma correction, and white balance adjustment or automatic whitebalance adjustment. The developing processor 46 performs secondadjustment regarding image settings based on characteristics of theimage, such as the wide-angle image, having undergone the firstadjustment. An example of the characteristics of the wide-angle image isan evaluative metered value of the image, such as the wide-angle image,which will be described later. Examples of the second adjustment includecontrast adjustment or automatic contrast adjustment, adjustment, orautomatic adjustment, of gamma correction, gain adjustment or automaticgain adjustment, and white balance adjustment or automatic white balanceadjustment. The second adjustment is performed on settings differentfrom settings of the first adjustment.

The developing processor 46 includes a preprocessor 50, a gain adjuster52, a white balance (WB) part 54, a demosaicing YRGB separator 56, anautomatic exposure (AE) preprocessor 58, an automatic white balance(AWB) preprocessor 60, a controller 62, a gamma converter 64, a colorcorrector 66, an edge enhancer 68, a noise reduction (NR) part 70, amemory 72, a dewarping part 74, a scaling part 76, a storage area 78, aWB part 80, a gamma-conversion/contrast adjuster 82, an image combiner84, an interface (I/F) unit part 86, and a cropping controller 88.

The preprocessor 50 acquires image data of a raw signal from the imagesensor 44 of the camera 20. The preprocessor 50 performs preprocessing,such as clamping, missing pixel correction, and lens shading correction,on the image data. The preprocessor 50 outputs the preprocessed imagedata to the gain adjuster 52.

The gain adjuster 52 adjusts brightness of the image data acquired fromthe preprocessor 50 by adjusting gain of the image data. The gainadjuster 52 outputs the gain-adjusted image data to the WB part 54.

The WB part 54 adjusts white balance of the image data acquired from thegain adjuster 52 by amplifying an R (Red) signal and a B (Blue) signalof the image data. The WB part 54 outputs the white-balanced image datato the demosaicing YRGB separator 56.

The demosaicing YRGB separator 56 applies demosaicing (interpolation) tothe image data acquired from the WB part 54, thereby allocating R, G(Green), and B signals on a pixel-by-pixel basis. The demosaicing YRGBseparator 56 calculates Y signals, which are luminance values, from theallocated RGB signals. The demosaicing YRGB separator 56 outputs the Ysignals to the AE preprocessor 58. The demosaicing YRGB separator 56outputs the RGB signals to the AWB preprocessor 60. The demosaicing YRGBseparator 56 outputs the RGB signals and the Y signals to the gammaconverter 64.

The AE preprocessor 58 acquires cropping control information, which isinformation about the cropped image of the speaker's area cropped fromthe wide-angle image, from the cropping controller 88. The AEpreprocessor 58 selects the speaker's area from the image data based onthe cropping control information and performs AE metering. The AEpreprocessor 58 calculates an AE evaluative metered value of the croppedspeaker's area from a result of the metering and outputs the AEevaluative metered value to the controller 62.

The AWB preprocessor 60 acquires the cropping control information fromthe cropping controller 88. The AWB preprocessor 60 selects thespeaker's area from the image data based on the cropping controlinformation and performs AWB metering. The AWB preprocessor 60calculates an AWB evaluative metered value of the cropped speaker's areabased on a result of the metering and outputs the AWB evaluative meteredvalue to the controller 62.

The controller 62 generates an exposure-time control signal, an aperturecontrol signal, a gain control signal, an RB control signal, a gammacontrol signal, and a contrast control signal based on the AE evaluativemetered value acquired from the AE preprocessor 58 and the AWBevaluative metered value acquired from the AWB preprocessor 60. Thecontroller 62 performs the first adjustment by feeding back thegenerated exposure-time control signal to the image sensor 44. Thecontroller 62 feeds back the generated aperture control signal to theaperture 42. The controller 62 feeds back the generated gain controlsignal to the gain adjuster 52. The controller 62 feeds back thegenerated RB control signal to the WB part 54. The controller 62 feedsback the generated gamma control signal to the gamma converter 64. Inresponse to the signals, the image sensor 44, the aperture 42, the gainadjuster 52, the WB part 54, and the gamma converter 64 adjustbrightness of the image data representing the wide-angle image so thatthe specified and cropped speaker's area achieves favorable brightness.

The gamma converter 64 performs gamma conversion of the RGB signalsacquired from the demosaicing YRGB separator 56 based on the gammacontrol signal acquired from the controller 62, and outputs theconverted RGB signals to the color corrector 66. The gamma converter 64performs gamma conversion of the Y signals acquired from the demosaicingYRGB separator 56 based on the gamma control signal acquired from thecontroller 62, and outputs the converted Y signals to the edge enhancer68.

The color corrector 66 adjusts hue, saturation, and the like of theimage data based on the RGB signals acquired from the gamma converter 64and, furthermore, corrects the image data based on spectral distributioncharacteristics of an infrared blocking filter. The color corrector 66outputs the corrected image data to the NR part 70.

The edge enhancer 68 performs processing of enhancing edges of thespeaker based on the Y signals, which are the luminance values, acquiredfrom the gamma converter 64. The edge enhancer 68 outputs the processedimage data to the NR part 70.

The NR part 70 reduces noises of the Y signals acquired from the edgeenhancer 68 and the RGB signals acquired from the color corrector 66.The NR part 70 writes the noise-reduced image data to the memory 72.

The dewarping part 74 acquires the cropping control information from thecropping controller 88. The dewarping part 74 reads out the image datarepresenting the cropped image of the speaker from the memory 72. Thedewarping part 74 performs dewarping on the read-out image datarepresenting the cropped image based on a dewarping map stored in thestorage area 78. The dewarping part 74 outputs the corrected image datarepresenting the cropped image to the scaling part 76.

The scaling part 76 performing an enlarging/reducing process on theimage data representing the cropped image acquired from the dewarpingpart 74. The scaling part 76 writes the image data representing theenlarged/reduced cropped image to the memory 72.

The AE preprocessor 58 reads out the Y signals of the image datarepresenting the wide-angle image having undergone the first adjustmentfrom the memory 72. The AE preprocessor 58 calculates an AE evaluativemetered value by performing AE metering on the read-out Y signals.

The AWB preprocessor 60 reads out the RGB signals of the image datarepresenting the wide-angle image having undergone the first adjustmentfrom the memory 72. The AWB preprocessor 60 calculates an AWB evaluativemetered value by performing AWB metering on the read-out RGB signals.

The controller 62 generates an RB control signal, a gain control signal,a gamma control signal, a contrast control signal, and the like based onthe AE evaluative metered value and the AWB evaluative metered value ofthe image data representing the wide-angle image and performs the secondadjustment. The controller 62 outputs the RB control signal to the WBpart 80. The controller 62 outputs the gamma control signal and thecontrast control signal to the gamma-conversion/contrast adjuster 82.

The WB part 80 reads out the image data representing the wide-angleimage from the memory 72. The WB part 80 adjusts white balance of theimage data representing the wide-angle image based on the RB controlsignal acquired from the controller 62. The WB part 80 outputs thewhite-balanced image data representing the wide-angle image to thegamma-conversion/contrast adjuster 82.

The gamma-conversion/contrast adjuster 82 performs, on the image datarepresenting the wide-angle image acquired from the WB part 80, gammaconversion and contrast adjustment based on the gamma control signal andthe contrast control signal acquired from the controller 62. Thegamma-conversion/contrast adjuster 82 writes the image data representingthe wide-angle image having undergone the gamma conversion and contrastadjustment to the memory 72.

The dewarping part 74 reads out the image data representing thewide-angle image written by the gamma-conversion/contrast adjuster 82from the memory 72. The dewarping part 74 performs dewarping on theread-out image data representing the wide-angle image based on thedewarping map, and thereafter outputs the dewarped image data to thescaling part 76.

The scaling part 76 performs an enlarging/reducing process on the imagedata representing the wide-angle image acquired from the dewarping part74, and thereafter writes the enlarged/reduced image data to the memory72.

The image combiner 84 reads out the image data representing the croppedimage and the image data representing the wide-angle image, eachrespectively having undergone the above-described image processing andthereafter written to the memory 72. The image combiner 84 generatescombined image data representing a single image, into which the imagedata representing the cropped image and the image data representing thewide-angle image are combined, and outputs the combined image data tothe I/F unit part 86.

The I/F unit part 86 converts the combined image data into serial imagedata (compliant with, e.g., the V-by-One (registered trademark) HSstandard or the HDMI (registered trademark) standard), and thereaftertransmits the serial image data to the server 12.

Because the developing processor 46 performs the first adjustment sothat the cropped image achieves favorable brightness and thereafterperforms the second adjustment on the wide-angle image in this way,adjusting the cropped image of the speaker to appropriate brightnesswhile adjusting the wide-angle image to appropriate brightness can beachieved.

FIG. 4 is a flowchart of a process for calculating an evaluative meteredvalue of a cropped image of a speaker. The evaluative metered valuecalculated in the flowchart corresponds to both the AE evaluativemetered value and the AWB evaluative metered value.

As illustrated in FIG. 4, in the process for calculating the evaluativemetered value of the cropped image, for each of pixel blocks of thecropped image of the speaker, the developing processor 46 adds up Ysignals, which are the luminance values, to calculate an average valueof the Y signals (S110).

FIG. 5 is a diagram describing a cropped image TP of a speaker. Thecropped image TP is generated by detecting a direction, in which aspeaker P0 is positioned, in a wide-angle image and cropping a partcorresponding to this direction from the wide-angle image. Referring tothe cropped image TP, the speaker P0 at the center is in front of awindow WD indicated as a dot-hatched area. A desk DS indicated as adiagonally-hatched area is arranged in front of the speaker P0. Thecropped image TP is backlit against the window WD, from which extraneouslight is coming. For this reason, the cropped image TP is such thatalthough the window WD is bright, the speaker P0 is too dark and detailsof the speaker P0 are not obvious. Taking this into account, a meteringmethod that places importance on the speaker P0 is used in the presentembodiment. Examples of the metering method usable in the presentembodiment include center-weighted metering that meters an image byplacing importance on a center portion of the image and spot meteringthat meters an image by placing importance on a specific part of theimage. The present embodiment is described through an example of thecenter-weighted metering.

The cropped image TP is divided into 6×6 pixel blocks BL. In FIG. 5,each of the pixel blocks BL is surrounded with dashed lines. The pixelblock BL contains, for example, 100×100 pixels. The developing processor46 calculates average values of Y signals of the cropped image TP bycalculating a total of Y signals of all the pixels contained in thepixel block BL and dividing the total by the number of pixels in thepixel block BL.

Referring back to FIG. 4, the developing processor 46 multiplies thecalculated average value of the Y signals by a weighting table WT foreach of the pixel blocks BL (S120). FIG. 6 is a diagram describing theweighting table WT for assigning weights to the Y signals of the croppedimage TP. The weighting table WT has weighting blocks WBL, into whichthe weighting table WT is divided as the cropped image TP is dividedinto the pixel blocks BL. Accordingly, in the present embodiment, theweighting block WBL is divided into 6×6. Weighting coefficients areassigned to the weighting blocks WBL. Because center weighted meteringis used in the present embodiment, large weighting coefficients areassigned to the weighting blocks WBL at a center portion. The developingprocessor 46 multiplies the average value of each of the pixel blocks BLby a weighting coefficient assigned to one, which corresponds to thepixel block BL, of the weighting blocks WBL.

Referring back to FIG. 4, the developing processor 46 calculates aweighted average value of the average values of the Y signals as anevaluative metered value of the cropped image TP (S130). Specifically,the developing processor 46 divides a total value of all the averagevalues, each multiplied by the corresponding weighting coefficient atS120, of the Y signals by a total value of all the weightingcoefficients, thereby calculating the weighted average value as theevaluative metered value of the cropped image TP. The evaluative meteredvalue of the cropped image TP is an example of the characteristics ofthe cropped image TP. Here, the process for calculating the evaluativemetered value performed by the developing processor 46 is completed.

FIG. 7 is a flowchart of an exposure control process performed on thecropped image TP of the speaker based on the evaluative metered value ofthe cropped image TP.

As illustrated in FIG. 7, in the exposure control process, thedeveloping processor 46 reads out the Y signals of the wide-angle imagefrom the memory 72 (S210). The developing processor 46 determines ato-be-metered area based on coordinate information of the cropped imageTP (S220). The to-be-metered area is such an area as that illustratedFIG. 5 where the cropped image TP is cropped in the wide-angle image.The developing processor 46 calculates an evaluative metered value ofthe cropped image TP in accordance with the above-described process forcalculating the evaluative metered value of the cropped image TP (S230).

The developing processor 46 calculates an error value, which is thedifference between the evaluative metered value of the cropped image TPand a predetermined target value, by making comparison between theevaluative metered value and the target value. As the target value, forexample, an evaluative metered value at an optimum exposure level may beapplied. The developing processor 46 determines whether or not the errorvalue is within a predetermined allowable range (S240). In other words,the developing processor 46 determines whether or not exposure of thecropped image TP is within an appropriate range. For example, thedeveloping processor 46 may determine whether or not the error value iswithin the allowable range by comparing the error value against apredetermined threshold value.

If it is determined that the error value is within the allowable range(Yes at S240), the exposure control process for the cropped image TPperformed by the developing processor 46 is completed. Thereafter, thedeveloping processor 46 may iterate the exposure control process for thecropped image TP.

If it is determined that the error value is not within the allowablerange (No at S240), the developing processor 46 determines whether ornot the error value is negative (S250). In other words, the developingprocessor 46 determines whether or not the cropped image TP is inunderexposure or overexposure.

If it is determined that the error value is negative (Yes at S250),because it is indicated the cropped image TP is in overexposure, thedeveloping processor 46 reduces an exposure level (S260). For example,the developing processor 46 reduces the exposure level by outputting acontrol signal for reducing the gain, a control signal for reducing theexposure time, and a control signal for narrowing the aperture 42 inorder. S260 is an example of the first adjustment. Thereafter, thedeveloping processor 46 may iterate the exposure control process for thecropped image TP.

If it is determined that the error value is not negative (No at S250),because it is indicated that the cropped image TP is in underexposure,the developing processor 46 raises the exposure level (S270). Forexample, the developing processor 46 raises the exposure level byoutputting a control signal for increasing the gain, a control signalfor increasing the exposure time, and a control signal for opening theaperture 42 wider in order. S270 is an example of the first adjustment.Thereafter, the developing processor 46 may iterate the exposure controlprocess for the cropped image TP.

Thus, because the developing processor 46 adjusts the exposure time, theaperture, and the gain by placing importance on the center portion wherethe speaker P0 is positioned in the cropped image TP rather than on aperipheral portion of the image, the developing processor 46 can adjustthe image of the speaker P0 to optimum brightness.

FIG. 8 is a flowchart of a process for calculating an evaluative meteredvalue of a wide-angle image WP. FIG. 9 is a diagram describing thewide-angle image WP.

Exposure of the wide-angle image WP illustrated in FIG. 9 is performedfor the purpose of determining the position of the speaker P0 relativeto the entire image and a condition of the wide-angle image WP in itsentirety. Exposure of the wide-angle image WP is preferably performedusing metering that makes the condition of the wide-angle image WPfavorable in its entirety. Examples of such favorable metering for thewide-angle image WP include average metering that calculates an averagevalue of Y signals (i.e., luminance values) of the entire wide-angleimage WP as an evaluative metered value and histogram metering thatobtains, as the evaluative metered value, a weighted average bycalculating and dividing a histogram and assigning weights to therespective pixel blocks BL divided in accordance with the dividedhistogram. The process for calculating the evaluative metered value ofthe wide-angle image presented in the flowchart illustrated in FIG. 8 isdescribed through an example of average metering.

As illustrated in FIG. 8, in the process for calculating the evaluativemetered value of the wide-angle image WP, the developing processor 46adds up Y signals, which are the luminance values, for each of the pixelblocks BL of the wide-angle image WP (S310).

The wide-angle image WP illustrated in FIG. 9 is an image of awide-angle area (e.g., an area of 360 degrees) taken with the camera 20.The wide-angle image WP contains a plurality of participants P1, P2, P3,P4, and P5. The participant P2, one of the participants P1 to P4, is thespeaker P0. The speaker P0 is in front of the window WD indicated as adot-hatched area. The desk DS indicated as a diagonally-hatched area isarranged in front of the participants P1 to P4. The cropped image TPdescribed above is the image inside a frame FR indicated by thick dashedlines.

The wide-angle image WP is divided into the 8×4 pixel blocks BL. In FIG.9, each of the pixel blocks BL is surrounded with dashed lines. Thepixel block BL contains, for example, 100×100 pixels. The developingprocessor 46 adds up Y signals of the 100×100 pixels of each of thepixel blocks BL.

The developing processor 46 calculates an average value of the Y signalsof each of the pixel blocks BL as an evaluative metered value of thewide-angle image WP (S320). Specifically, the developing processor 46calculates the average value of the Y signals by dividing a total valueobtained by adding up the Y signals by the number of pixels in each ofthe pixel blocks BL. The evaluative metered value of the wide-angleimage WP is an example of the characteristics of the wide-angle imageWP. Here, a control process for calculating the evaluative metered valueof the wide-angle image WP performed by the developing processor 46 iscompleted.

FIG. 10 is a flowchart of an exposure control process performed on thewide-angle image WP based on the evaluative metered value of thewide-angle image WP.

As illustrated in FIG. 10, in the exposure control process, thedeveloping processor 46 reads out the Y signals of the wide-angle imageWP from the memory 72 (S410). The developing processor 46 calculates anevaluative metered value of the wide-angle image WP in accordance withthe above-described process for calculating the evaluative metered valueof the wide-angle image WP (S420).

The developing processor 46 calculates an error value, which is thedifference between the evaluative metered value of the wide-angle imageWP and a predetermined target value, by making comparison between theevaluative metered value and the target value. As the target value, forexample, an evaluative metered value at an optimum exposure level may beapplied. The developing processor 46 determines whether or not the errorvalue is within a predetermined allowable range (S430). In other words,the developing processor 46 determines whether or not exposure of thewide-angle image WP is within an appropriate range. For example, thedeveloping processor 46 may determine whether or not the error value iswithin the allowable range by comparing the error value against apredetermined threshold value.

If it is determined that the error value is within the allowable range(Yes at S430), the exposure control process for the wide-angle image WPperformed by the developing processor 46 is completed. Thereafter, thedeveloping processor 46 may iterate the exposure control process for thewide-angle image WP.

If it is determined that the error value is not within the allowablerange (No at S430), the developing processor 46 determines whether ornot the error value is negative (S440). In other words, the developingprocessor 46 determines whether or not the wide-angle image WP is inunderexposure or overexposure.

If it is determined that the error value is negative (Yes at S440),because it is indicated the wide-angle image WP is in overexposure, thedeveloping processor 46 reduces the exposure level (S450). For example,the developing processor 46 reduces the exposure level by outputting acontrol signal for darkening contrast and a control signal for setting agamma correction value to a value smaller than 1. S450 is an example ofthe second adjustment. Thereafter, the developing processor 46 mayiterate the exposure control process for the wide-angle image WP.

If it is determined that the error value is not negative (No at S440),because it is indicated that the wide-angle image WP is inunderexposure, the developing processor 46 raises the exposure level(S460). For example, the developing processor 46 raises the exposurelevel by outputting a control signal for brightening contrast and acontrol signal for setting the gamma correction value to a value largerthan 1. S460 is an example of the second adjustment. Thereafter, thedeveloping processor 46 may iterate the exposure control process for thewide-angle image WP.

FIG. 11 is a histogram describing an example of exposure control for thewide-angle image WP. FIG. 12 is a histogram stretched to a region whereY signals (luminance values) are low in the exposure control for thewide-angle image WP.

When the exposure control illustrated in FIG. 4 and FIG. 7 is applied toan image taken in such a condition where the speaker P0 is backlit asillustrated in FIG. 5, because exposure settings are adjusted suitablefor the speaker P0 that is dark, the other areas of the image than thespeaker P0 are adjusted to be too bright. In this case, as illustratedin FIG. 11, frequencies in a region (region encircled with a dashedline) where Y signals (luminance values) are low are small andfrequencies concentrate to a region where Y signals are high. Bystretching the histogram illustrated in FIG. 11 to the region where theY signals are low, such a histogram as illustrated in FIG. 12 wherefrequencies are distributed also to the region where the Y signals arelow can be obtained. By applying exposure control to the wide-angleimage WP based on this histogram, the wide-angle image WP withappropriate exposure can be generated. Example methods for stretchingthe histogram include contrast adjustment, gamma correction, and bitshift.

FIG. 13 is a diagram describing an example of the method for stretchingthe histogram. For example, as illustrated in FIG. 13, the histogram canbe stretched to the region where the Y signals are low by pulling thecurve illustrated in FIG. 13 to a region where contrast is dark asindicated by arrow AR1 and deforming the curve so as to reduce a gammacorrection value to a value smaller than 1 as indicated by arrow AR2. Asa result, appropriate exposure can be achieved also in the wide-angleimage WP.

Functions, linkages, and the like of or between elements of theabove-described embodiment can be modified as appropriate.

In the above description, the video delivery system 10 according to theembodiment is applied to a video conferencing system. Alternatively, theembodiment may be applied to other video delivery systems includingremote monitoring systems.

A video delivery system according to an aspect of the present inventionis advantageously capable of displaying a cropped image with appropriatesettings.

The above-described embodiments are illustrative and do not limit thepresent invention. Thus, numerous additional modifications andvariations are possible in light of the above teachings. For example, atleast one element of different illustrative and exemplary embodimentsherein may be combined with each other or substituted for each otherwithin the scope of this disclosure and appended claims. Further,features of components of the embodiments, such as the number, theposition, and the shape are not limited the embodiments and thus may bepreferably set. It is therefore to be understood that within the scopeof the appended claims, the disclosure of the present invention may bepracticed otherwise than as specifically described herein.

The method steps, processes, or operations described herein are not tobe construed as necessarily requiring their performance in theparticular order discussed or illustrated, unless specificallyidentified as an order of performance or clearly identified through thecontext. It is also to be understood that additional or alternativesteps may be employed.

Further, any of the above-described apparatus, devices or units can beimplemented as a hardware apparatus, such as a special-purpose circuitor device, or as a hardware/software combination, such as a processorexecuting a software program.

Further, as described above, any one of the above-described and othermethods of the present invention may be embodied in the form of acomputer program stored in any kind of storage medium. Examples ofstorage mediums include, but are not limited to, flexible disk, harddisk, optical discs, magneto-optical discs, magnetic tapes, nonvolatilememory, semiconductor memory, read-only-memory (ROM), etc.

Alternatively, any one of the above-described and other methods of thepresent invention may be implemented by an application specificintegrated circuit (ASIC), a digital signal processor (DSP) or a fieldprogrammable gate array (FPGA), prepared by interconnecting anappropriate network of conventional component circuits or by acombination thereof with one or more conventional general purposemicroprocessors or signal processors programmed accordingly.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. Processing circuitryincludes a programmed processor, as a processor includes circuitry. Aprocessing circuit also includes devices such as an application specificintegrated circuit (ASIC), digital signal processor (DSP), fieldprogrammable gate array (FPGA) and conventional circuit componentsarranged to perform the recited functions.

What is claimed is:
 1. A video delivery terminal comprising: an imagecapturer configured to perform image capture to generate an image of asubject; and a developing processor configured to crop an area of aportion of the image to generate a cropped image, perform firstadjustment regarding the cropped image based on characteristics of thecropped image, and perform second adjustment regarding the imageincluding the cropped image having undergone the first adjustment,wherein the first adjustment includes adjusting at least one of exposuretime and an aperture of the image capturer.
 2. A video delivery terminalcomprising: an image capturer configured to perform image capture togenerate an image of a subject; and a developing processor configured tocrop an area of a portion of the image to generate a cropped image,perform first adjustment regarding the cropped image based oncharacteristics of the cropped image, and perform second adjustmentregarding the image including the cropped image having undergone thefirst adjustment, wherein the first adjustment and the second adjustmentinclude at least one of gain adjustment, gamma correction adjustment,and contrast adjustment.
 3. A video delivery terminal comprising: animage capturer configured to perform image capture to generate an imageof a subject; and a developing processor configured to crop an area of aportion of the image to generate a cropped image, perform firstadjustment regarding the cropped image based on characteristics of thecropped image, and perform second adjustment regarding the imageincluding the cropped image having undergone the first adjustment,wherein the first adjustment and the second adjustment include whitebalance adjustment.
 4. The video delivery terminal according to claim 1,wherein the second adjustment is performed on settings different fromsettings of the first adjustment.
 5. A non-transitory computer-readablemedium comprising computer readable program codes, performed by aprocessor, the program codes when executed causing the processor toexecute: performing image capture to generate an image of a subject;cropping an area of a portion of the image to generate a cropped image;performing first adjustment regarding the cropped image based oncharacteristics of the cropped image; and performing second adjustmentregarding the image including the cropped image having undergone thefirst adjustment, wherein the first adjustment includes adjusting atleast one of exposure time and an aperture of the image capturer.
 6. Avideo delivery method comprising: performing image capture to generatean image of a subject; cropping an area of a portion of the image togenerate a cropped image; performing first adjustment regarding thecropped image based on characteristics of the cropped image; andperforming second adjustment regarding the image including the croppedimage having undergone the first adjustment, wherein the firstadjustment includes adjusting at least one of exposure time and anaperture of the image capturer.
 7. A non-transitory computer-readablemedium comprising computer readable program codes, performed by aprocessor, the program codes when executed causing the processor toexecute: performing image capture to generate an image of a subject;cropping an area of a portion of the image to generate a cropped image;performing first adjustment regarding the cropped image based oncharacteristics of the cropped image; and performing second adjustmentregarding the image including the cropped image having undergone thefirst adjustment, wherein the first adjustment and the second adjustmentinclude at least one of gain adjustment, gamma correction adjustment,and contrast adjustment.
 8. A non-transitory computer-readable mediumcomprising computer readable program codes, performed by a processor,the program codes when executed causing the processor to execute:performing image capture to generate an image of a subject; cropping anarea of a portion of the image to generate a cropped image; performingfirst adjustment regarding the cropped image based on characteristics ofthe cropped image; and performing second adjustment regarding the imageincluding the cropped image having undergone the first adjustment,wherein the first adjustment and the second adjustment include whitebalance adjustment.
 9. A video delivery method comprising: performingimage capture to generate an image of a subject; cropping an area of aportion of the image to generate a cropped image; performing firstadjustment regarding the cropped image based on characteristics of thecropped image; and performing second adjustment regarding the imageincluding the cropped image having undergone the first adjustment,wherein the first adjustment and the second adjustment include at leastone of gain adjustment, gamma correction adjustment, and contrastadjustment.
 10. A video delivery method comprising: performing imagecapture to generate an image of a subject; cropping an area of a portionof the image to generate a cropped image; performing first adjustmentregarding the cropped image based on characteristics of the croppedimage; and performing second adjustment regarding the image includingthe cropped image having undergone the first adjustment, wherein thefirst adjustment and the second adjustment include white balanceadjustment.